Lecture 7 : Decision - making under uncertainty : Part 1

This lecture is an introduction to decision theory, which gives tools for making rational choices in face of uncertainty. It is useful in all kinds of disciplines from electrical engineering to economics. In computer science, a compelling setting to consider is an autonomous vehicle or robot navigating in a new environment. It may have some prior notions about the environment but inevitably it encounters many different situations and must respond to them. The actions it chooses (drive over the object on the road or drive around it?) changes the set of future events it will see, and thus its choice of the immediate action must necessarily take into account the continuing effects of that choice far into the future. You can immediately see that the same issues arise in any kind of decision-making in real life: save your money in stocks or bonds; go to grad school or get a job; marry the person you are dating now, or wait a few more years? Of course, italicized terms in the previous paragraph are all very loaded. What is a rational choice? What is “uncertainty”? In everyday life uncertainty can be interpreted in many ways: risk, ignorance, probability, etc. Decision theory suggests some answers —perhaps simplistic, but a good start. The theory has the following three elements. The first element is its probabilistic interpretation of uncertainty: there is a probability distribution on future events, and furthermore, the decision maker knows this distribution. The second element is how it quantifies what the decision-make wants: he/she derives some utility from the events that happen. Utility is a number that satisfies some intuitive axioms such as monotonicity and concavity (look it up on wikipedia). The third element of the theory is its definition of a “rational choice.” The decision-making is said to be rational if it maximises the expected utility.